<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="Author" content="Jeff W. Boote">
   <meta name="GENERATOR" content="Mozilla/4.79 [en] (Windows NT 5.0; U) [Netscape]">
   <title>BWCTL</title>
<!--									-->
<!--	$Id$	-->
<!--									-->
<!------------------------------------------------------------------------>
<!--									-->
<!--			   Copyright (C)  2004				-->
<!--				Internet2				-->
<!--			   All Rights Reserved				-->
<!--									-->
<!------------------------------------------------------------------------>
<!--									-->
<!--	File:		index.html					-->
<!--									-->
<!--	Author:		Jeff Boote					-->
<!--			Internet2					-->
<!--									-->
<!--	Date:		Sat Feb 07 18:33:19 MST 2004			-->
<!--									-->
<!--	Description:							-->
<!--	Overview documentation for bwctl.				-->
<!--									-->
</head>
<body>

<center>
<h1>BWCTL Version 1.4.2rc3 </h1></center>

<center>(<b>B</b>and<b>w</b>idth <b>C</b>on<b>t</b>ro<b>l</b>)
<br><a href="http://e2epi.internet2.edu/bwctl">http://e2epi.internet2.edu/bwctl/</a></center>

<center>
<h5>
$Date$</h5></center>
BWCTL is a command line client application and a scheduling and policy
daemon that wraps the throughput testing tools <b><a
href="http://sourceforge.net/projects/iperf">Iperf</a></b>, <b><a
href="http://e2epi.internet2.edu/thrulay/">Thrulay</a></b>, and <b><a
href="http://www.lcp.nrl.navy.mil/nuttcp/">Nuttcp</a></b>. These tests can measure
maximum TCP bandwidth, with various tuning options available, or, by doing a
UDP test, the delay, jitter, and datagram loss of a network.
<p>
The <b>bwctl</b> client application works by contacting a <b>bwctld</b> process
on the two test endpoint systems.  BWCTL will work as a 3-party application.
The client can arrange a test between two servers on two different systems. If
the local system is intended to be one of the endpoints of the test, <b>bwctl</b>
will detect whether a local <b>bwctld</b> is running and will handle the required server
functionality if needed. <b>bwctld</b> manages and schedules the resources of the host
on which it runs. A more detailed description of the architecture is available
in the <a href="architecture.html">architecture documentation</a>.
<p>The <b>bwctl</b> client is used to request the type of throughput
test wanted. Furthermore, it requests <i>when</i> the test should be executed. <b>bwctld</b>
either responds with a tentative reservation or a test denied message.
Once <b>bwctl</b> is able to get a matching reservation from both <b>bwctld</b>
processes (one for each host involved in the test), it confirms the reservation.
Then, the <b>bwctld</b> processes run the test and return the results.
The results are returned to the client from both sides of the test. Additionally,
the <b>bwctld</b> processes share the results from their respective sides
of the test with each other.
<p>BWCTL is used to enable non-specific throughput tests to a host without
having to give full user accounts on the given system. Users want the ability
to run throughput tests to determine the achievable or available bandwidth
between a pair of hosts. It is often useful to test to multiple points
along a network path to determine the network characteristics along that
path. Typically, users who want to do this path decomposition have to contact
the network/system administrators that control the hosts along the path
directly. The administrator needs to either run half of the test for the
user or provide a user account on the host. Also, network paths of interest
usually are controlled by multiple administrators. These hurdles have made
this kind of testing difficult in practice.
<p>BWCTL was designed to help with this problem. It allows an administrator
to configure a given host as an <i>Iperf</i>, <i>Thrulay</i> or <i>Nuttcp</i>
endpoint that can be shared by multiple users without concern that those users
will interfere with each other. Specific policy limits can be applied to
specific users, and individual tests are scheduled so they will not interfere
with each other.  BWCTL allows the administrator to classify incoming
connections based upon a user name and AES key (generated by a pass phrase)
combination or, alternatively, based upon an IP/netmask. Once the connection is
classified, the <b>bwctld</b> can determine the exact type and intensities of
throughput tests that will be allowed. (More details on the policy controls
can be found in the <a href="bwctld.limits.man.html">bwctld.limits(8)</a>
manual page.)
<h3>

<hr WIDTH="100%">What's Changed Since Version 1.2a</h3>

<ul>
    <li>Allows for different throughput testing tools. Currently supported are <b><a href="http://dast.nlanr.net/Projects/Iperf/">Iperf</a></b>, <b><a href="http://thrulay-hd.sourceforge.net/">Thrulay</a></b> and <b><a
href="http://www.lcp.nrl.navy.mil/nuttcp/">Nuttcp</a></b>. Use the -T option on bwctl to specify which throughput testing tool to use. Use bwctl -h to see which are supported.</li>
    <ul>
        <li>NOTE: Some of the throughput test options are not available for some of the testers. BWCTL does not support UDP tests, changing the output format or changing the output units with either <b>Nuttcp</b> or <b>Thrulay</b>.</li>
    </ul>
    <li>Allows throughput tests to use multiple, parallel streams. This can be specified with the -P option in the bwctl client.</li>
    <li>Allows for UDP tests with bandwidth limits of higher than 4.3Gbps. This only works with 1.3 clients and servers.</li>
    <li>Allows for comma-separated value output for <i>Iperf</i>. Add "-y c" to the bwctl command line client to use it.</li>
    <li>Allows administrators to define scripts to run at the completion of each test, allowing for results to be stored in a database, accounting tracking or any number of other tasks.</li>
    <li>Improves robustness of time/scheduling.</li>
    <li>Improves error handling of <i>Iperf</i> children processes.</li>
    <li>Improves error messages for failure conditions.</li>
</ul>

<hr WIDTH="100%">What's Changed Since Version 1.1b</h3>

<ul>
    <li>Added -f option to bwctld to allow it to run with root permissions.</li>
    <li>Added better usage() messages and more information to several user prompts.</li>
    <li>Fixed bug that caused all server side errors to be reported incorrectly be the client.</li>
    <li>Fixed bwctld bug that disabled srcnode option.</li>
    <li>Ported to Solaris.</li>
    <li>Ported to OS X.</li>
    <li>Removed NTP system call dependency. Added allow_unsync options.</li>
    <li>Increased default -L value for bwctl.</li>
    <li>Added messages to data output to indicate if <i>Iperf</i> was killed.</li>
    <li>Modified all config options to have '_' between words.</li>
</ul>

<h3>

<hr WIDTH="100%">Features</h3>

<ul>
<li>
Supports version 2.0.x of <b>Iperf</b>.</li>

<li>
Supports version 5.3.1 of <b>Nuttcp</b>.</li>
<ul>
        <li>NOTE: BWCTL does not support UDP tests, changing the output format or changing the output units with <b>Nuttcp</b>.</li>
</ul>

<li>
Includes built-in support for <b>Thrulay</b>.</li>
<ul>
        <li>NOTE: BWCTL does not support UDP tests, changing the output format or changing the output units with <b>Thrulay</b>.</li>
</ul>

<li>
Full IPv6 support. No options needed. If the target of a test is specified
by a DNS hostname, and that name has both an IPv4 and an IPv6 address defined,
the <b>bwctl</b> command line application prefers the IPv6 address.</li>

<li>
Data from both sides of the test is returned so that sending rate can be
compared with receiving rate.</li>

<li>
3-Party communication is supported. The client does not have to be on one
of the test endpoint hosts.</li>

<li>
A local <b>bwctld</b> is not required. <b>bwctl</b> will detect if there
is a local <b>bwctld</b> and use it if it is there; <b>bwctl</b> will mimic the
missing functionality if needed.</li>

<li>
Port ranges for the <b>bwctld</b> peer connections can be specified to
allow <b>bwctld</b> to work in firewall environments.</li>

<li>
Port ranges for the throughput tests can be specified to allow passive
monitoring to characterize the test streams more easily.</li>

<li>
Limits the types of throughput tests that can be run.  Sending a set amount of
data is not supported because BWCTL needs to know how long the test will take
before the test is started.</li>

<li>
Supports dynamic TCP window size determination. (Currently, this is limited
because the bottleneck capacity is specified as a constant in the config
file. In the future, bottleneck capacity could potentially be dynamically
determined using techniques such as packet dispersion.)</li>
</ul>

<h3>

<hr WIDTH="100%">Requirements</h3>

<ul>
<li>BWCTL includes the <b>Thrulay</b> tester, but it would be best if <b>Iperf</b> 2.0.x is installed on the system.</li>

<li>
The <b>bwctld</b> daemon prefers an NTP synchronized system clock to ensure the
two endpoints of the test are actually agreeing to the same scheduled time
window for test execution.
Therefore, BWCTL requires that <b>NTP</b> be running to synchronize the
system clock. (See the <em>allow_unsync</em> configuration option
described in the <a href="bwctld.conf.man.html">bwctld.conf(8)</a> manual page
to see how to by-pass this requirement.)
<b>NTP</b> should be setup correctly on the system so that <b>bwctld</b>
can actually fetch reasonable estimates of the time error from <b>NTP</b>.
For the <b>NTP</b> algorithms to work correctly,
<b>NTP <font color="#FF0000">MUST</font></b><font color="#000000">
be configured with no fewer than 4 clocks. (See</font> <a href="http://twiki.ntp.org/bin/view/Support/SelectingOffsiteNTPServers">http://twiki.ntp.org/bin/view/Support/SelectingOffsiteNTPServers</a>&nbsp;<font color="#000000">
for more details.)</font></li>

<li>
<b>gnumake</b> may be required for the build process (see
<a href="#Build">Building the Application</a>).</li>

<li>
To get good results, the endpoints will need to be well
tuned. See <a href="http://www.psc.edu/networking/projects/tcptune/">http://www.psc.edu/networking/projects/tcptune/</a>
and <a href="http://www-didc.lbl.gov/TCP-tuning/buffers.html">http://www-didc.lbl.gov/TCP-tuning/buffers.html</a>
for some excellent tips in that regard.</li>
</ul>

<h3>

<hr WIDTH="100%">Supported Systems</h3>
<p>BWCTL has been tested most extensively on Red Hat Linux 4 and 5 systems, but
most recent versions of UNIX should work. The code cleanly compiles on FreeBSD,
Linux, and OS X.</p>

<p>BWCTL compiles on Solaris. However, <i>Thrulay</i> does not work on that
platform so is not compiled in. If anyone has any interest in updating thrulay
to work on Solaris, let us know.</p>

<h3>
<hr WIDTH="100%">Recommended Hardware</h3>
BWCTL does not have any strict hardware requirements. The limits imposed
by the specific throughput tests are more taxing to hardware than the
scheduling and policy functions added by BWCTL. The hardware requirements
are therefore a function of the specific throughput tests that are desired.
<b>Internet2</b>
had good luck using the following hardware to perform 1Gbps tests on the
<b>Abilene</b> network:
<dl>
<dd>
Intel SCB2 motherboard</dd>

<dd>
2 x 1.266 GHz PIII, 512 KB L2 cache, 133 MHz FSB</dd>

<dd>
2 x 512 MB ECC registered RAM (one/slot to enable interleaving)</dd>

<dd>
2 x Seagate 18 GB SCSI (ST318406LC)</dd>

<dd>
SysConnect Gigabit Ethernet SK-9843 SX</dd>
</dl>

<h3>

<hr WIDTH="100%"><a NAME="Build"></a>Building the Application</h3>
The BWCTL software uses the <i>gnu autoconf</i> tools to configure the
build process. BWCTL is distributed with a pre-built <b>configure</b> script
so the actual <i>autoconf</i> tools should not be needed on the target
system. (Although, <b>gnumake</b> may be required.) The
<b>configure</b>
script can be run with the <i>--help</i> option to determine the full set
of configurable options.
<p>A basic build procedure would be:
<blockquote>
<pre>% gzip -cd bwctl-$VERS.tar.gz | tar xf -
% cd bwctl-$VERS
&nbsp; # --prefix is only needed if you don't like the default
&nbsp; #&nbsp;&nbsp; (/usr/local on most systems)
% ./configure --prefix=/install/root
% make
% make install</pre>
</blockquote>
Please report any build problems to <a href="mailto:bwctl-users@internet2.edu">bwctl-users@internet2.edu</a>.
<h3>

<hr WIDTH="100%">Configuring <i>bwctld</i></h3>
The basic procedure to configure <b>bwctld</b> is to create a
<a href="bwctld.conf.man.html">bwctld.conf</a>
and, optionally, a <a href="bwctld.limits.man.html">bwctld.limits</a> file
and a
<a href="bwctld.keys.man.html">bwctld.keys</a> file. These files
need to be installed in the same directory that is specified with the <b>-c</b>
option to <b>bwctld</b>. The recommended directory is <i>/install/root/etc</i>.
(The <i>etc</i> directory below your install root.) There are examples
of these files in the <i>bwctl-$VERS/conf&nbsp;</i> sub directory of the
distribution.
<p><b>bwctld.conf:</b>
<br>Used to configure basic operation of the daemon, such as server listening
port, the path for <b>Iperf</b>, and error logging. For a detailed description
of the options available, see the <a href="bwctld.conf.man.html">bwctld.conf(8)</a>
manual page.
<p><b>bwctld.limits:</b>
<br>Used to configure the policy limits for the daemon. There are two parts
to this policy: 1) authentication and 2) authorization. The authentication
is done by assigning a class to each new connection. Each class has a set
of limits associated with it. The classes are hierarchical, so a connection
must pass the limit restrictions of the assigned class as well as all parent
classes. For a detailed description of the options available, see the
<a href="bwctld.conf.man.html">bwctld.limits(8)</a>
manual page.
<p><b>bwctl.keys:</b>
<br>Used to associate identities with AES keys. These identities are then
mapped to a class by the <b>bwctld.limits</b> file. For a more detailed
description, see the
<a href="bwctld.keys.man.html">bwctld.keys(8)</a>
manual page.

<hr WIDTH="100%"><h3>Configuring firewalls</h3>
If a firewall is enabled on the machine running bwctld, it will need to be
configured to allow incoming clients and tests. Use
<a href="bwctl_firewalls.html">this set of directions</a> to configure the
firewall.

<hr WIDTH="100%"><h3>Configuring NTP</h3>
To ensure that tests can be run, the machine running bwctld should have its
clock synchronized using NTP.  Use <a href="bwctl_ntp_conf.html">this set of
directions</a> to configure NTP.

<hr WIDTH="100%"><h3>Running <i>bwctld</i></h3>
The normal command line to start the daemon is:
<blockquote>
<pre>&nbsp;% bwctld -c /install/root/etc</pre>
</blockquote>
It is possible to run the daemon without a config file directory if enough
command line options are specified, but it is easier to use a config file.
<p>To see all the available options:
<blockquote>
<pre>&nbsp;% bwctld -h</pre>
</blockquote>
More details on running the daemon, as well as a complete description of
the command line options, is available from the <a href="bwctld.man.html">bwctld</a>
manual page.
<h3>

<hr WIDTH="100%">Running <i>bwctl</i></h3>
The basic command line for the client is:
<blockquote>
<pre>% bwctl [options] [-c catchhost] [-s sendhost]</pre>
</blockquote>
The <i>-c</i> option indicates that the <i>catchhost</i> should be an
<b>Iperf</b>
server (think packet <b>c</b>atcher). The <i>-s</i> option indicates the
<i>sendhost</i>
should be an <b>Iperf</b> client (think packet <b>s</b>ender). At least
one of the -c/-s options must be specified. If only one of these options
is specified, the local host is assumed to be the other test endpoint.
<p>To see a list of available options:
<blockquote>
<pre>% bwctl -h</pre>
</blockquote>
More details on running the client application, with a complete description
of all command line options, is available from the <a href="bwctl.man.html">bwctl</a>
manual page.
<h3>

<hr WIDTH="100%">Caveats</h3>
Thrulay support is deprecated. If there is enough desire, or if someone is
willing to maintain thrulay, the functionality could be undeprecated.

<hr WIDTH="100%">Mailing Lists</h3>
There are two email lists to support this software:
<dl>
<dt>
<b>bwctl-users</b></dt>

<dd>
This is a general discussion list for users to discuss problems. 
This list is monitored as bug reports should be sent here.
</dd>

<dt>
<b>bwctl-announce</b></dt>

<dd>
This list is used to announce new versions or significant developments
with regard to the software.</dd>
</dl>
Information about these lists, including links to subscribe, is at
<a href="https://mail.internet2.edu/wws/lists/engineering">https://mail.internet2.edu/wws/lists/engineering</a>.

<h3>
<hr WIDTH="100%">Author</h3>
Jeff Boote
<br><a href="mailto:boote@internet2.edu">boote@internet2.edu</a>
<br>Internet2
<br><br>
Aaron Brown
<br><a href="mailto:aaron@internet2.edu">aaron@internet2.edu</a>
<br>Internet2

<h3><hr WIDTH="100%">Acknowledgments</h3>
The following individuals have greatly aided in the development, testing, or
debugging of BWCTL:
<ul>
<li>Shumon Huque (UPenn)</li>
<li>Paul Hyder (NOAA)</li>
<li>Warren Matthews (GaTech)</li>
<li>Sean Peisert (SDSC)</li>
<li>Chris Tracy (MAX)</li>
</ul>

<pre>
<hr WIDTH="100%">$Id$</pre>

</body>
</html>
